Composable Abstractions for Deep Learning
Deep learning is an approach to AI that has become enormously popular in recent years. It involves training neural networks, composed of many layers, to learn the structure of a given data set. Deep learning involves a variety of concepts: Automatic differentiation, weights and biases, gradient-based optimization and so on.
For this topic, we are interested in useful programming abstractions that capture the essence of deep learning and provide a composable foundation for implementing neural networks. In particular, Cruttwell et al. have observed that parametric lenses unify a number of concepts in deep learning. Parametric lenses capture the fact that deep learning is parametrized, as results are determined by weights and hyperparameters, and bidirectional, as information flows both forwards (to compute results) and backwards (to update the model's weights).
In the seminar, you will
- Provide a brief introduction to deep learning using an existing framework like Keras or Equinox,
- Show how lenses can be used in programming using a library like Monocle,
- Investigate the approach by Cruttwell et al., and
- Explain how it can be used by programmers to write deep learning algorithms.
In the project, you will write a small deep learning framework in a statically-typed, functional programming language, making use of the abstractions featured in Cruttwell et al. or other works.